Import Libraries¶
In this project, some necessary libraries are needed to be imported.
import warnings
warnings.filterwarnings('ignore')
import tensorflow as tf
from tensorflow import keras
from keras import models, layers, utils, losses, optimizers, initializers, regularizers
from keras.models import *
from keras.layers import *
from keras.utils import np_utils, image_dataset_from_directory, plot_model
from keras.callbacks import EarlyStopping, ReduceLROnPlateau
from sklearn.metrics import confusion_matrix, classification_report
import random
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import plotly.express as px
!pip install git+https://github.com/paulgavrikov/visualkeras
import visualkeras
Read the txt File Specifying the Image File Locations¶
The flist.txt file specifies the locations of all the images.
flist = list(pd.read_csv('/kaggle/input/dissertation-1-data/Files/flist.txt', header = None)[0])
Overview of the properties.csv and properties_juny12.csv Datasets¶
Load datasets properties and properties_juny12.csv.
properties = pd.read_csv('/kaggle/input/dissertation-1-data/Files/properties.csv')
properties_juny12 = pd.read_csv('/kaggle/input/dissertation-1-data/Files/properties_juny12.csv')
Combine their rows, and make sure that propertyType is a categorical variable.
properties_full = pd.concat([properties, properties_juny12])
properties = properties_full
properties.propertyType = properties.propertyType.astype('category')
Show the basic information of the dataset.
properties.info()
There are entries in total, and no missing values are found for the variable propertyType.
Select a Subset of the Whole Dataset¶
Extract the identity numbers from the list flist.
flist_id = list(map(lambda string: string[-40 : -4], flist))
Take the entries in the full dataset that have a street view image attached.
properties_sub = pd.DataFrame(properties.loc[properties['property_id'].isin(flist_id)])
Discard entries linking with the same images, so that the images linking with each identity numbers are unique. These entries are the ones having the same latitude and longitude values.
properties_sub = properties_sub.drop_duplicates(['location_lat', 'location_lng'])
Make sure that propertyType is a categorical variable.
properties_sub.propertyType = properties_sub.propertyType.astype('category')
Extract the identity numbers again, and show the basic information of the new dataset.
flist_id = list(properties_sub.property_id)
properties_sub.info()
From the new dataset, we could see that there are entries, and there are no missing values for the vaiable propertyType.
Plots for Exploratory Analysis¶
Geometric Scatter Plot of the Locations of Properties¶
The locations of the properties are shown on the map as follows. You could zoom in and zoom out, and if you hover your cursor over a point, it would show you the location information, as well as the detailed address.
px.scatter_geo(properties_sub,
lat = 'location_lat',
lon = 'location_lng',
scope = 'europe',
hover_name = 'address').show()
Types of properties are listed, and the numbers in each type are calculated.
labels = list(properties_sub.propertyType.cat.categories)
counts = np.array(properties_sub.propertyType.value_counts(sort = False))
Pie Chart of the Types of Properties¶
A pie chart is then plotted, showing the proportions of different types of properties in the new dataset.
plt.title('Pie Chart of the Types of Properties')
plt.pie(counts, labels = labels, explode = [.1, .1, .1, .1, .1], shadow = True, autopct = '%1.1f%%')
plt.show()
From the pie chart, we could see that this is an imbalanced dataset.
Load the Image Data¶
Specify the directory, the sizes of images, the number of images to be loaded in one batch, and the random seed. The seed is a random integer ranging from to , the largest number that Kaggle could store. The purpose of setting such a random seed is to ensure that for each time, the dataset is being split randomly.
From this step, the images related to the Unknown class of property types are removed.
directory = '/kaggle/input/dissertation-1-data/Files/street_view/'
height = 64
width = 64
batch = 32
seed = random.randint(1, np.power(2, 32) - 1)
Train-Validation-Test Split¶
Split the dataset into training, validation, and testing sets. The proportions of them are , , and . Split the training and validation sets first on a - basis, and split the validation set on a - basis by using skip and take. Cache and Prefetch methods are applied to save runtime and memory. One hot encoder for the labels are introduced by setting label_mode = 'categorical'.
tr_val = image_dataset_from_directory(
directory,
validation_split = .3,
subset = 'both',
seed = seed,
image_size = (height, width),
batch_size = batch,
label_mode = 'categorical')
training = tr_val[0].cache().shuffle(1000).prefetch(buffer_size = tf.data.AUTOTUNE)
validation = tr_val[1].cache().prefetch(buffer_size = tf.data.AUTOTUNE)
val_bat = tf.data.experimental.cardinality(validation)
validation = validation.skip((2 * val_bat) // 3).cache().prefetch(buffer_size = tf.data.AUTOTUNE)
testing = validation.take((2 * val_bat) // 3).cache().prefetch(buffer_size = tf.data.AUTOTUNE)
Model Components Settings¶
The actual types of properties in the training and testing sets are extracted first, and the labels specify the types of properties. They are detached, flat, semi-detached, and terraced.
propertyType_train_fac = np.argmax(np.asarray(list(training.unbatch().map(lambda x, y: y))), axis = 1)
propertyType_test_fac = np.argmax(np.asarray(list(testing.unbatch().map(lambda x, y: y))), axis = 1)
labels = pd.Series(propertyType_train_fac).astype('category')
Assign Class Weights to the Categorical Cross Entropy Loss Function¶
In our models, categorical cross entropy loss is applied. Since the dataset is imbalanced, the class weights should be taken into account. They are calculated as follows. For each class, the product of the number of classes and the number of samples in this class is calculated at first. The weight for this class is represented by the ratio between the total number of samples and the product mentioned above.
loss = losses.CategoricalCrossentropy()
weights = sum(labels.value_counts()) / (labels.value_counts(sort = False) * len(labels.cat.categories))
loss.weighted = weights
Allow Callbacks to Monitor the Models¶
The models should be terminated from training when the validation accuracy is not getting any higher for 5 epochs.
The learning rate has an initial value of . Every time the validation accuracy decreases, or when it fails to reach the highest value in previous epochs, the learning rate is multiplied by .
lr = 5e-4
Callbacks = [
EarlyStopping(monitor = 'val_accuracy', mode = 'max', patience = 5, restore_best_weights = True),
ReduceLROnPlateau(monitor = 'val_accuracy', factor = 4e-2, patience = 0, cooldown = 0)
]
Visualising Model Accuracy and Model Loss¶
def visualise_accuracy_loss(history):
'''
This function takes the training and validation results of a Keras sequential model as an input,
and sketches 2 plots showing the model accuracy and model loss.
'''
# extract training and validation accuracy
ac = history.history['accuracy']
val_ac = history.history['val_accuracy']
# extract training and validation loss
lo = history.history['loss']
val_lo = history.history['val_loss']
# a series of epochs
epoch = len(ac)
epoch_vec = np.arange(epoch) + 1
# plot model accuracy
plt.figure(figsize = (10, 10))
plt.subplot(1, 2, 1)
plt.plot(epoch_vec, ac, label = 'Training Accuracy')
plt.plot(epoch_vec, val_ac, label = 'Validation Accuracy')
plt.legend(loc = 'lower right')
plt.title('Model Accuracy')
# plot model loss
plt.subplot(1, 2, 2)
plt.plot(epoch_vec, lo, label = 'Training Loss')
plt.plot(epoch_vec, val_lo, label = 'Validation Loss')
plt.legend(loc = 'upper right')
plt.title('Model Loss')
plt.show()
Modelling¶
Since the components of the models have been set up, we are able to build models.
Multi-Layer Perceptron Model¶
mlp = Sequential([
Rescaling(1. / 255, input_shape = (height, width, 3)), # normalise the pixel values
Flatten(), # reshape them into a vector
Dense(160, activation = tf.nn.leaky_relu), # fully connected layer with 160 neurons, uses leaky relu
Dense( # output layer, 4 neurons, uses softmax, L1-regularisation techniques applied
len(labels.cat.categories),
activation = tf.nn.softmax,
kernel_initializer = initializers.RandomNormal(),
bias_initializer = initializers.Zeros(),
kernel_regularizer = regularizers.L1(1e-5),
bias_regularizer = regularizers.L1(10),
activity_regularizer = regularizers.L1(10)
)
])
# uses weighted loss, adam optimiser, and accuracy as evaluation metric
mlp.compile(loss = loss, optimizer = optimizers.Adam(learning_rate = lr), metrics = ['accuracy'])
# visualise the layers
visualkeras.layered_view(mlp, legend = True)
Show the structure of the model.
plot_model(mlp, show_shapes = True, show_layer_names = True, show_layer_activations = True)
Train and validate the model. Visualise the accuracy and loss.
history = mlp.fit(training, validation_data = validation, epochs = 60, batch_size = 50, callbacks = Callbacks)
visualise_accuracy_loss(history)
Calculate the confusion matrix of the property type in the testing set, and visualise it using a heat map.
propertyType_test_pred_MLP = np.argmax(mlp.predict(testing), axis = 1)
label_names = ['Detached', 'Flat', 'Semi-Detached', 'Terraced']
confmat_MLP = pd.DataFrame(confusion_matrix(propertyType_test_fac, propertyType_test_pred_MLP))
ax = plt.axes()
sns.heatmap(confmat_MLP, annot = True,
cmap = 'YlGnBu', fmt = 'g',
cbar = True, xticklabels = label_names,
yticklabels = label_names, ax = ax)
ax.set_xlabel('Predicted Classes')
ax.set_ylabel('True Classes')
ax.set_title('Confusion Matrix of Property Types Predicted by the MLP Model')
plt.show()
Report necessary classification results in the testing set.
print(
classification_report(
propertyType_test_fac,
propertyType_test_pred_MLP,
target_names = label_names
)
)
Convolutional Neural Network Model¶
cnn = Sequential([
Rescaling(1. / 255, input_shape = (height, width, 3)), # normalise the image data
Conv2D(50, 4, padding = 'same', activation = tf.nn.leaky_relu), # convolutional layer, uses leaky relu
MaxPooling2D(pool_size = (2, 2)), # 2 by 2 max pooling
Conv2D(48, 4, padding = 'same', activation = tf.nn.leaky_relu), # convolutional layer, uses leaky relu
MaxPooling2D(pool_size = (2, 2)), # 2 by 2 max pooling
Flatten(), # flatten the matrix
Dense(42, activation = tf.nn.leaky_relu), # fully connected layer, uses leaky relu
Dropout(.25), # dropping out neurons in order to avoid overfitting
Dense( # output layer, 4 neurons, uses softmax, L1-regularisation techniques applied
len(labels.cat.categories),
activation = tf.nn.softmax,
kernel_initializer = initializers.RandomNormal(),
bias_initializer = initializers.Zeros(),
kernel_regularizer = regularizers.L1(1e-5),
bias_regularizer = regularizers.L1(10),
activity_regularizer = regularizers.L1(10)
)
])
# uses weighted loss, adam optimiser, and accuracy as evaluation metric
cnn.compile(loss = loss, optimizer = optimizers.Adam(learning_rate = lr), metrics = ['accuracy'])
# visualise the layers
visualkeras.layered_view(cnn, legend = True)
Show the structure of the model.
plot_model(cnn, to_file = 'cnn.png', show_shapes = True, show_layer_names = True, show_layer_activations = True)
Train and validate the model. Visualise the accuracy and loss.
history = cnn.fit(training, validation_data = validation, epochs = 60, batch_size = 50, callbacks = Callbacks)
visualise_accuracy_loss(history)
Calculate the confusion matrix of the property type in the testing set, and visualise it using a heat map.
propertyType_test_pred_CNN = np.argmax(cnn.predict(testing), axis = 1)
confmat_CNN = confusion_matrix(propertyType_test_fac, propertyType_test_pred_CNN)
ax = plt.axes()
sns.heatmap(confmat_CNN, annot = True,
cmap = 'YlGnBu', fmt = 'g',
cbar = True, xticklabels = label_names,
yticklabels = label_names, ax = ax)
ax.set_xlabel('Predicted Classes')
ax.set_ylabel('True Classes')
ax.set_title('Confusion Matrix of Property Types Predicted by the CNN Model')
plt.show()
Report necessary classification results in the testing set.
print(
classification_report(
propertyType_test_fac,
propertyType_test_pred_CNN,
target_names = label_names
)
)